Energy efficient electrical systems and methods for modular data centers and modular data pods

ABSTRACT

An efficient, modular, direct current (DC) uninterruptible power supply (UPS) for at least one server of a data center is disclosed. The single-conversion DC UPS includes an AC-DC converter, an energy storage device electrically coupled to the output of the AC-DC converter, and a single conversion server supply DC-DC converter electrically coupled to the AC-DC converter and the energy storage device, which may be a low-voltage lithium-ion battery or combined with an ultra capacitor. The DC UPS may be incorporated into a UPS system for a data center including a plurality of server rack assemblies and a plurality of cooling distribution units (CDUs). The UPS system includes an electric generator, an AC UPS electrically coupled between the electric generator and the plurality of CDUs, and a plurality of DC UPSs coupled between the electric generator and the plurality of server rack assemblies.

BACKGROUND 1. Technical Field

This present disclosure provides a unique design solution using a directcurrent (DC) uninterruptible power supply (UPS) and server power supplywith DC input voltage for the design of high-efficiency, cost-effectivemodular data centers.

The present disclosure also provides unique design solutions for highdensity modular data centers by using either: (i) an alternating current(AC) uninterruptible power supply (UPS) in energy saver (ES) mode; or(ii) a direct current (DC) UPS in conjunction with a server power supplywith a DC input voltage.

2. Background of Related Art

There is a large demand for efficient Data Centers to store largeamounts of data due to the emergence of Web 2.0-enabled businesses suchas financial, e-commerce, pharmaceutical, or multi-media businesses.Indeed, information technology (IT) growth is outstripping Moore's law,which is a rule of thumb where the number of transistors that caninexpensively be placed in an integrated circuit (IC) doublesapproximately every two years. However, in the past many years, majorenergy efficiency improvements and technological innovation has not beenachieved for both electrical and mechanical infrastructure of the datacenter industry, even as computing hardware and software has become muchmore efficient. Thus, efficient computing hardware and software sits onan inefficient electrical and mechanical infrastructure. Thisinefficient infrastructure represents significant capital expenditure(CAPEX) and operational expenditure (OPEX) cost problems for businesses.Data centers and businesses with mission critical data storagerequirements need a high-efficient and reliable infrastructure systemsto reduce overall total cost of ownership (TCO).

By some estimates, the demand for data storage doubles approximatelyevery 18 months, which results in an annual growth rate of approximately150% for the next 5 years. This increase in demand for data storage islinearly proportional to the increase in the amount of data beingprocessed, which is being driven by the increase in mobile data trafficand data processing of large, medium, and small business operations.

Modularity and flexibility are key elements in allowing for a datacenter to grow and change over time. A modular data center may consistof data center equipment contained within shipping containers or similarportable containers. But it can also be described as a design style inwhich components of the data center are prefabricated and standardizedso that they can be constructed, moved or added to quickly as needschange.

The digital storage market doubles every 18 months, which translates toan annual growth rate of approximately 150% for the next 5 years.Standard brick-and-mortar data centers require significant capitalexpenditure (CAPEX) and operational expenditure (OPEX), which causeproblems for businesses. However, modular data centers requireapproximately 50% lower construction costs in comparison to standardbrick-and-mortar data centers. Also, modular data centers can bedeployed within 12 to 15 weeks in comparison to 2 years needed bystandard brick-and-mortar data centers. Existing businesses in need ofadditional data storage require highly efficient and reliable modulardata centers to reduce the overall total cost of ownership (TCO).

Various companies including IO, HP, Dell, Google, and Colt supplymodular data centers. However, these modular data centers are in somerespects rudimentary with respect to energy efficiency. Moreover, thecooling mechanisms of these modular data centers are also rudimentaryand inefficient with respect to energy efficiency. In addition, theelectrical infrastructure is inefficient or does not have a redundantsupply path. Most electrical infrastructure uses standard shippingcontainers for storage applications. Since it assumed in the art thatefforts at improving efficiency will result in a decrease inreliability, comparatively little attention has been focused onimproving the power efficiency of the data centers while at the sametime maintaining the same or even greater levels of reliability.

With respect to the large conventional modular data centers, FIG. 1Ashows an existing data center 10 with a centralized alternating current(AC) UPS 40, i.e., a double conversion AC-DC/DC-AC, and doubleconversion AC-DC/DC-DC server power supplies 45 a-45 n with an AC inputand a DC output. Information technology (IT) loads 55 and mechanicalloads 60 (e.g., the energy required by cooling systems) of the datacenter 10 are powered entirely by a utility feed 20 via an on-linedouble conversion AC-UPS 40 through a step-down transformer 35. Agenerator 15 starts to operate once a disturbance in the utility 20,e.g., a loss of all or a portion of the electricity provided by theutility feed 20, is more than approximately two seconds. During adisturbance in the utility feed 20, surge protector 25 dampens thedisturbance. For disturbances beyond a pre-determined acceptable levelthat are beyond the dampening capabilities of the surge protector 25,the IT loads 55 and mechanical loads 60 are powered by the AC UPS 40 viaone or more internal batteries and an internal DC-AC inverter section,neither of which are shown but are part of AC UPS 40.

Once the generator 15 has reached its reference speed and stabilized,the transfer switch 30 shifts the primary power source from the utilityfeed 20 to the generator 15. Thereafter, the IT loads 55 and mechanicalloads 60 are entirely powered by the generator 15 via the on-line UPS40. The internal batteries (not shown) of AC UPS 40 are also rechargedby the generator 15. Once the disturbance in the utility feed 20 is nolonger present, the IT loads 55 and mechanical loads 60 are shifted fromthe generator 15 to the UPS system 40. Ultimately, the transfer switch30 shifts the primary power source from the UPS system 40 back to theutility feed 20.

A mechanical cooling system (not shown) is in thermal communication witheach of a plurality of IT server racks 50 a-50 n, and circulates acoolant that removes heat generated by the plurality of IT server racks50 a-50 n. The coolant is pumped by a cooling distribution unit (CDU) 65a-65 n that includes a heat exchanger that allows the system to userefrigerant cooling. Each CDU 65 a . . . 65 n may support approximately350 kW of IT load capacity, i.e., part of the mechanical load 60.

Turning now to FIGS. 2 and 3, there is illustrated a data center 100having AC UPSs 145 a, 145 b and a server power supply with an AC input.During normal operation, the IT or server loads 50 ( 50 a . . . 50 n)and mechanical loads 65 a-1 . . . 65 n-1 and 65 a-2 and 65 n-1 of datacenter 100 (see FIG. 3) are powered entirely by the utility powerfeeders 105 a, 105 b (see FIG. 2) via on-line double conversion AC UPSs145 a, 145 b that are similar to the centralized AC UPS 40 describedabove with respect to FIG. 1A. The utility power source is usuallyconnected to the data center 100 through the first utility power feeder105 a. The second utility power feeder 105 b is normally open at switch105′ and supplies the data center load (both IT and mechanical loads 55,60 as described above with respect to FIG. 1A) in case the first utilitypower feeder 105 a malfunctions. The utility power feeders 105 a and 105b supply power through switchgear 110 to feed step-down transformer 115.The voltage output of step-down transformer 115 is supplied to a firstcommon bus 120. Power feed 122, in turn, supplies power from the firstcommon bus 120 to a second common bus 135.

In turn, power feed 150 supplies power from the second common bus 135 tofirst primary common bus 150 a via branch bus line 1501 and to secondprimary common bus 150 b via branch bus line 1502. Power from firstprimary common bus 150 a is supplied in turn to first secondary commonbus 162 a and also to second secondary common bus 162 b via feeds 152 aand 152 b, respectively. Similarly, power from second primary common bus150 b is supplied in turn to first secondary common bus 162 a and alsoto second secondary common bus 162 b via feeds 154 a and 154 b,respectively.

Power is supplied to transformers 170 a and 170 c from first secondarycommon bus 162 a via feed 164 a and split feed 164 a 1 to transformer170 a and via split feed 164 a 2 to transformer 170 c. Similarly, poweris supplied to transformers 170 b and 170 d from second secondary commonbus 162 b via feed 164 b and split feed 164 b 1 to transformer 170 b andvia split feed 164 b 2 to transformer 170 d. Power is supplied directlyfrom first secondary common bus 162 a via feed 166 a to transformer 170e and directly from secondary common bust 162 b via feed 166 b totransformer 170 f.

When a disturbance in the utility power feeders 105 a, 105 b occurs thatis more than about two seconds, the generators 140 a, 140 b start. Thedisturbance is detected in bus 135 and generator 140 a supplies power tofeed 1351 through AC UPS 145 a while generator 140 b supplies power tofeed 1352 through AC UPS 145 b. AC UPS 145 a then supplies power to bus150 a via branch bus feeder 13511 and to bus 150 b via branch bus 13512.Similarly, AC UPS 145 b then supplies power to bus 150 a via branch busfeeder 13521 and to bus 150 b via branch bus 13522.

In some cases, only one of generators 140 a and 140 b may startdepending on the magnitude of the IT or server loads 50 (50 a . . . 50n) and mechanical loads 65 a-1 . . . 65 n-1 and 65 a-2 . . . 65 n-1.When the generators 140 a, 140 b start, the IT and mechanical loads arestill powered by the UPSs 145 a, 145 b via inverter 430 and battery 410(see FIG. 4). When the generators 140 a, 140 b have reached theirreference speeds and stabilized, the transfer switch (not shown) shiftsthe primary power source from the utility power feeder 105 a to thegenerators 140 a, 140 b. Thereafter, the loads IT or server loads 50 (50a . . . 50 n) and mechanical loads 65 a-1 . . . 65 n-1 and 65 a-2 . . .65 n-2 are entirely powered by the generators 140 a, 140 b via the UPSs145 a, 145 b. The UPS batteries 410 (See FIG. 4) are recharged by powergenerators 140 a, 140 b. When the disturbance in the utility powerfeeder 105 a is no longer present, the loads 55 and 60 are shifted fromthe generators 140 a, 140 b to the UPSs 145 a, 145 b and ultimatelytransfer switch shifts the primary power source to the utility powerfeeder 105 a. Transformer 115 steps down the voltage from the utilityfeed 105 a.

An auxiliary distribution source 160 and transformer 165, which areelectrically coupled in series via feed bus 1601 and then via split feedbus 16011 with first primary common bus 150 a and then via split feedbus 16012 with second primary common bus 150 b, supply power to theloads IT or server loads 50 (50 a . . . 50 n) and mechanical loads 65a-1 . . . 65 n-1 and 65 a-2 . . . 65 n-2 upon failure of the UPSs 145 a,145 b and/or generators 140 a, 140 b. Wrap up lines 125 a, 125 b, whichare electrically coupled to first common bus 120, provide an alternativepath for supplying power to the loads if a problem arises in the AC UPSs145 a, 145 b and/or generators 140 a, 140 b. Wrap up lines 125 a and 125b include switches 130 a and 130 b, respectively, which are normallyopen and provide power if other lines electrically coupled to mainswitch gear 120 fail. Switch gears 162 a and 162 b allow for either ofthe UPSs 145 a, 145 b and/or either of the generators 140 a, 140 b tosupply all or a part of the power for the entire IT or server loads 50(50 a . . . 50 n) and mechanical loads 65 a-1 . . . 65 n-1 and 65 a-2 .. . 65 n-2 (see FIG. 1A).

Transformers 170 c, 170 d supply power to IT or server loads 50 (50 a .. . 50 n) via switches 185 a and 185 b, respectively. If eithertransformer 170 c or 170 d fails, then tie 187 assists in supplyingpower to bus 190 a or 190 b. Mechanical transformers 170 a, 170 b supplypower to mechanical (CDU) loads 65 a-1 . . . 65 n-1 and 65 a-2 . . . 65n-2 via switches 175 a and 175 b, respectively (see FIG. 1A). If eithertransformer 170 a or 170 b fails, then tie 177 assists in supplyingpower to bus 180 a or 180 b. Administration transformers 170 e, 170 fsupply power to an administration building load (not shown) via switches195 a and 195 b respectively. If either transformer 170 e or 170 ffails, then tie 197 assists in supplying power to the administrationbuilding load.

Either transformer 170 a or 170 b has sufficient capacity to handle theentire mechanical (CDU) loads 65 a-1 . . . 65 n-1 and 65 a-2 . . .65 n-2alone in case of failure of the other. However, transformers 170 a and170 b generally work in combination, each carrying 50% of the load.Similarly, either transformer 170 c or 170 d has sufficient capacity tohandle the entire IT or server loads 50 (50 a . . . 50 n) alone in caseof failure of the other. However, transformers 170 c and 170 d generallywork in combination, each carrying 50% of the load. Buses 190 a and 190b are for IT or server loads 50 (50 a . . . 50 n) and buses 180 a and180 b are for mechanical/CDU loads 65 a-1 . . . 65 n-1 and 65 a-2 . . .65 n-2.

FIG. 3 shows connections between buses 180 a, 180 b and mechanical/CDUloads 65 a-1 . . . 65 n-1 and 65 a-2 . . . 65 n-2 and between buses 190a, 190 b and IT load 50. More particularly, mechanical CDU loads 65 a-1. . . 65 n-1 are supplied power from bus 180 a and CDU loads 65 a-2 . .. 65 n-2 are supplied power from bus 180 b.

IT load 50 includes a plurality of server rack assemblies 50 a . . . 50n, which are separated from each other to define hot aisles 210 and coldaisles 220. Each server rack assembly 50 a . . . 50 n is electricallycoupled to buses 190 a and 190 b via server power supplies 500 (see FIG.5). Each server power supply 500 includes an AC-DC converter 520 a . . .520 n electrically coupled in series to a DC-DC converter 510 a . . .510 n having outputs 530 a, 530 b, 530 c, 530 d electrically coupled tothe respective IT or server loads 50 a . . . 50 n.

FIG. 4 is a block diagram of an AC UPS 400 that can be used as the ACUPSs 145 a, 145 b of FIG. 2. The AC UPS 400 includes an AC-DC converter440, a DC-DC converter 420, a battery 410, and a DC-AC inverter 430. Asshown, the DC-DC converter 420 and the battery 410 are electricallycoupled in parallel between the AC-DC converter 440 and the DC-ACinverter 430. The AC-DC converter 440 receives a high AC voltage via aplurality of power lines 460 a . . . 460 c and converts the high ACvoltage to a high DC voltage. The high DC voltage is supplied to boththe bidirectional DC-DC converter 420 and the DC-AC inverter 430. TheDC-DC converter 420, which is a buck-boost converter, steps down thehigh DC voltage to a lower voltage that is suitable for charging leadacid battery 410 (e.g., an intermediate voltage).

In this case, high DC voltage is defined as about 1000 V and anintermediate voltage for the battery 410 is about 300 V DC to about 600V DC. The three-phase inverter 430 also includes three outputs, that is,3-phase 480 V AC outputs 450 a, 450 b, 450 c. It should be noted thatthe individual 3-phase 480 V AC outputs 450 a, 450 b, 450 c are notexplicitly shown in FIGS. 2 and 3 but are represented as single phaselines in a single line diagram. The UPS efficiency in double-conversionmode is around 94%-96% at nominal load.

During normal operation, the DC-AC inverter 430 converts the high DCvoltage from the AC-DC converter 440 into an AC voltage, which issupplied to the server power supplies 500 via step-down transformers 170c, 170 d of FIG. 2. When there is a disturbance in the high AC voltagesupplied by a utility to the AC-DC converter 440, the DC-DC converter420 converts the voltage of the battery 410 into a high DC voltage,which is supplied to the DC-AC inverter 430. The AC UPS 400 is adouble-conversion AC UPS because it performs two electrical conversionsvia the AC-DC converter 440 and the DC-AC inverter 430.

FIG. 5 shows server power supply 500, which includes two AC inputs(single phase) 540 a, 540 b from respective IT buses 190 a, 190 b (seeFIG. 3). The AC-DC converter 520 converts a single phase AC voltage ofthe AC inputs 540 a, 540 b to an intermediate DC voltage. DC-DCconverter 510 converts the intermediate DC voltage into multiple low DCvoltages 530 a . . . 530 d. For example, the plurality of DC-DCconverters 510 a . . . 510 n can supply approximately 3.3 VDC, 5 VDC, 12VDC, and −12 VDC via phase output lines 530 a, 530 b, 530 c, and 530 d,respectively, to the respective servers 50 a . . . 50 n as shown in FIG.3.

One disadvantage of the existing data center 100 is thedouble-conversion AC UPS 400. The two electrical conversions performedby the AC UPS 400 increase losses and increase power usage effectiveness(PUE) of the data center. PUE is a measure of how efficiently a datacenter uses its power. Specifically, PUE is a measure of how much of thepower is actually used by the servers of the data center in contrast tothe power used for cooling and other overhead functions of the datacenter. In other words, PUE is the ratio of the total amount of powerused by a data center to the power delivered to the servers of the datacenter so that PUE is greater than 1.0, which is the ideal PUE value.Thus, the lower the PUE, the more efficient is the data center.

Another disadvantage of the existing data center 100 is the multipleelectrical conversions (AC-DC/DC-DC) performed by the server powersupply 500 (FIG. 5), which also increases losses and increases PUE.Therefore, the overall losses introduced by the AC UPS 400 and theserver power supply 500 are high in the existing data center 100.

Alternatively, FIG. 1B shows a data center system 70 including amodular, scalable, double-conversion UPS system 72 and server powersupplies with AC input voltages. More particularly, the server powersupplies of the modular UPS system 72 include a plurality of in-linemodular AC UPSs 75 a . . . 75 n that are each electrically coupledbetween the transformer 35 and a respective one of the plurality ofserver power supplies 45 a . . . 45 n. The modular UPS system 72 alsoincludes a plurality of in-line AC UPSs 85 a . . . 85 n that are eachelectrically coupled between the transformer 35 and a respective one ofthe plurality of CDUs 65 a . . . 65 n. The efficiency of the modular UPSsystem 72 is high because of the higher loading factor of the modularUPS system 72 of FIG. 1B as compared to the centralized UPS system 40 ofFIG. 1A, where the initial loading factor may not be high. Normally, thecapacity of the centralized UPS system 40 of FIG. 1A is selected basedon future load demand.

FIG. 6 illustrates an existing modular data center 1100 powered by autility feed 1110 (e.g., 480 V L-L 3-phase AC utility feed). Multiple ITor server loads 1140 a . . . 1140 n are supplied by a 3-phase doubleconversion on-line UPS 1120. DC power is supplied to the IT loads 1140 a. . . 1140 n from, for example, 277 V L-N, 1-phase double conversionAC-DC/DC-DC converters 1130 a . . . 1130 n. The single phase (L-N) issupplied via the UPS 1210. The DC-DC output of the server power suppliesor converters 1130 a . . . 1130 n is supplied to the IT loads 1140 a . .. 1140 n. A cooling mechanism 1115 (or mechanical load) is supplied bythe utility feed 1110 (e.g., 480 V L-L 3-phase AC).

The cooling mechanism 1115 may employ Computer Room Air Conditioning(CRAC) cooling which is a central unit within the modular data center100 that is relatively inefficient.

FIG. 7 illustrates another existing modular data center 1200 powered byutility feed 1110 (e.g., 480 V L-L 3-phase AC). The AC inputs of theserver power supplies 1130 a . . . 1130 n are powered by a plurality ofrack mountable 1-phase double conversion on-line AC UPSs 1120 a . . .1120 n. In other words, each server power supply 1130 a . . . 1130 n isdirectly connected to an AC UPS 1120 a . . . 1120 n, in contrast to thesystem 1100 illustrated in FIG. 6. As such, every IT load 1140 a . . .1140 n is powered by separate and distinct AC UPSs 1120 a . . . 1120 n.The system 1200 may also include a CRAC cooling mechanism 1115, asdescribed above with reference to FIG. 6.

FIG. 8 illustrates a block diagram of 1-phase double conversion on-lineAC UPSs 1120 a . . . 1120 n of FIG. 7. AC UPSs 1120 a . . . 1120 n eachincludes a 1-phase AC-DC converter 1310, a DC-DC converter 1320, abattery 1330, and a DC-AC 1-phase inverter 1340. As compared to the ACUPS 400 in FIG. 4, the AC UPS 400 is a 3-phase version of adouble-conversion AC UPS whereas AC UPSs 1120 a . . . 1120 n are singlephase versions of the double conversion AC UPS 400. The 3-phase AC UPS400 has a larger power rating and cannot be mounted on the server racks50 a . . . 50 n. However, 1-phase AC UPSs 1120 a . . . 1120 n may bemounted on the server rack 50 a . . . 50 n as they have a smaller powerrating.

As shown in FIG. 8, the DC-DC converter 1320 and the battery 1330 areelectrically coupled in parallel between the AC-DC converter 1310 andthe DC-AC inverter 1340. The AC-DC converter 1310 receives a 1-phasemedium AC voltage (e.g., 277 V AC) via a plurality of power lines 1405a, 1405 b and converts the medium AC voltage to a medium DC voltage. Themedium DC voltage is supplied to both the bidirectional DC-DC converter1320 and the 1-phase DC-AC inverter 1340. The inverter 1340 alsoincludes two outputs, that is, outputs 1445 a, 1445 b. The DC-DCconverter 1320, which is a buck-boost converter, steps down the mediumDC voltage to a lower voltage that is suitable for charging the leadacid battery (e.g., an intermediate voltage).

In view of foregoing, there are multiple conversions required for thesupply of electrical power in conventional modular data centers andmodular data pods.

SUMMARY

In one aspect, the systems, and corresponding methods, of the presentdisclosure relate to a modular system for supplying DC power to at leastone server. The modular system includes a DC uninterruptible powersupply (UPS) including: an AC-DC converter and an energy storage deviceelectrically coupled to the output of the AC-DC converter. The modularsystem also includes a DC-DC converter directly connected to the outputof the AC-DC converter of the DC UPS and configured to supply DC powerto the at least one server.

The energy storage device may be a low-voltage battery. The low-voltagebattery may be a 12 V battery, a 24 V battery, or a 48 V battery. Theenergy storage device may be a lithium-ion battery or a combination of alithium-ion battery and a ultra-capacitor.

The DC-DC converter may supply a plurality of DC voltages to the atleast one server. The single power conversion may be performed by aserver power supply between the energy storage device and the at leastone server.

In another aspect, the present disclosure features a system forsupplying power to a data center including a plurality of server rackassemblies and a plurality of cooling distribution units (CDUs) inthermal communication with the plurality of server rack assemblies. Thesystem includes an electric generator, an AC uninterruptible powersupply (UPS) electrically coupled in energy storage mode between theelectric generator and the plurality of CDUs and a plurality of DC UPSs,each of which is electrically coupled between the electric generator anda respective one of the plurality of server rack assemblies.

Each of the plurality of DC UPSs may include an AC-DC converter and anenergy storage device electrically coupled in parallel to the output ofthe AC-DC converter. Each of the plurality of DC UPSs may furtherinclude a high-frequency DC-DC converter for a power supply of at leastone server in a respective server rack assembly, and the high-frequencyDC-DC converter may include a plurality of MOSFETs in an H-bridgeconfiguration and a zero-voltage switching controller electricallycoupled to the plurality of MOSFETs to output a plurality of DC voltagesfrom the high-frequency DC-DC converter. The high frequency may bebetween about 80 kHz to about 120 kHz.

The energy storage device may be a low-voltage battery. The low-voltagebattery may be a 12 V battery, a 24 V battery, or a 48 V battery. Theenergy storage device may be a lithium-ion battery or a combination of alithium-ion battery and an ultra-capacitor.

The AC UPS may be configured in an offline energy saver mode such thatpower is supplied from the AC UPS to the plurality of CDUs if adisturbance occurs in a utility power source that normally suppliespower to the plurality of CDUs. A single power conversion may beperformed between the energy storage device and the at least one serverin the respective server rack assembly.

Each of the plurality of DC UPSs may be connected to a respective loadline of a plurality of load lines. One of the plurality of DC UPSs maybe connected to one load line of a plurality of load lines.

In another aspect, the AC UPS in ES mode includes: an AC-DC converter,an energy storage device, and a bidirectional DC-DC converterelectrically coupled in series with the positive terminal of the energystorage device. The series combination of the energy storage device andthe DC-DC converter is coupled in parallel to the AC-DC converter. TheAC UPS in ES mode also includes a DC-AC inverter electrically coupled inparallel to the series combination of the energy storage device and theDC-DC converter.

The battery/energy storage device may be a medium voltage battery. Themedium voltage battery may supply a voltage between 250 V and 450 V.

Each of the plurality of DC UPSs may be connected to a respective loadline of a plurality of load lines. One of the plurality of DC UPSs maybe connected to one load line of a plurality of load lines.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows a conventional modular data center with a centralizeddouble conversion (AC-DC/DC-AC) alternating current AC UPS, and doubleconversion (AC-DC/DC-DC) server power supplies with an AC input and a DCoutput supplying IT and mechanical loads;

FIG. 1B shows a conventional modular data center system includingmodular, scalable, double-conversion UPS systems and server powersupplies with AC input voltages;

FIG. 2 is a schematic, one-line illustration of the bus powerarrangement of a conventional modular data center having AC UPSs and aserver power supply with an AC input voltage in which, during normaloperation, IT and mechanical loads of the data center are poweredentirely by the utility power feeders via on-line double conversion ACUPSs that are similar to the centralized AC UPS illustrated above withrespect to FIG. 1A;

FIG. 3 is a schematic, one-line illustration of the connections betweenbuses dedicated to mechanical/CDU loads and connections between busesdedicated to IT loads for the conventional modular data center of FIG.2;

FIG. 4 is a schematic block diagram of a 3-phase double-conversion ACUPS that can be used as the AC UPSs of FIG. 2, which includes an AC-DCconverter, a bi-directional DC-DC converter for a battery, a battery asan energy storage device, and a DC-AC 3-phase inverter;

FIG. 5 is a schematic illustration of a prior art single-phase serverpower supply, which includes two AC inputs (1-phase AC) from respectiveIT buses, that converts an AC voltage of the AC inputs to anintermediate DC voltage and, in turn, into multiple low DC voltagesusing its DC-DC converters;

FIG. 6 is a schematic illustration of a conventional modular data centerpowered by a 3-phase AC utility feed (400 or 480 V AC) and centralizedAC UPS;

FIG. 7 illustrates another conventional modular data center powered by autility feed wherein each IT load is powered by separate and distinctmodular single-phase AC UPSs;

FIG. 8 illustrates a block diagram of a single-phase double conversionon-line AC UPS as illustrated in FIG. 7;

FIG. 9 shows a data center according to embodiments of the presentdisclosure having modular single-conversion DC UPSs and a modularsingle-conversion DC-DC server power supply wherein CDUs are connectedto AC UPSs and the AC UPSs are connected in an efficient energy saver(ES) mode;

FIG. 10A is a schematic, one-line illustration of the bus powerarrangement of a modular data center according to one embodiment of thepresent disclosure having DC UPSs and a server power supply with DCinput voltage as shown in FIG. 9 in which, during normal operation, ITand mechanical loads of the data center are able to be powered entirelyby the utility power feeders through a DC UPS;

FIG. 10B is a schematic illustration of the connections between busesdedicated to mechanical/CDU loads and connections between busesdedicated to IT loads for the modular data center of FIG. 10B accordingto one embodiment of the present disclosure as shown in FIG. 9;

FIG. 11 illustrates a DC UPS which includes a battery and an AC-DCconverter which includes AC inputs from each of the cableways of FIG.10B according to one embodiment of the present disclosure wherein theoutputs from AC-DC converter are supplied to the battery and the DC-DCconverter of the server power supply;

FIG. 12 illustrates a DC-DC converter which includes two DC inputs fromrespective bus feeds and which includes multiple outputs for supplying aplurality of different DC voltages to the servers illustrated in FIG.10B according to one embodiment of the present disclosure;

FIG. 13 is a schematic diagram of a DC-DC converter according to oneembodiment of the present disclosure in which a DC voltage is suppliedfrom either a battery or the output of the AC-DC converter illustratedin FIG. 12 to a DC-DC converter in which a zero-voltage switching (ZVS)control strategy is applied;

FIG. 14 shows a modular data center system including modular, scalable,UPS systems and server power supplies with AC input voltages similar tothe data center system of FIG. 1B wherein AC UPSs are configured tooperate in off-line energy saver (ES) mode;

FIG. 15 illustrates a modular data pod schematic according to oneembodiment of the present disclosure wherein an AC UPS is connected inoff-line energy saver (ES) mode in which utility power from one utilityfeed bus is supplied directly to critical IT loads and mechanical loadsduring normal operation;

FIG. 16 illustrates the modular data center according to one embodimentof the present disclosure, where each utility power feeder line hasseparate and distinct UPSs connected in off-line energy saver (ES) mode;

FIG. 17A is a schematic illustration of one embodiment of the presentdisclosure of the bus power arrangement of a modular data pod havingoff-line double conversion AC UPSs in energy saver mode and a serverpower supply with an AC-DC converter and a DC-DC converter in which,during normal operation, IT and mechanical loads of the data center arepowered entirely by the utility power feeders as illustrated in FIG. 16;

FIG. 17B is a continuation of the schematic illustrations of oneembodiment (FIG. 17A) of the present disclosure of the bus powerarrangement of a modular data pod having off-line double conversion ACUPSs in an energy saver mode of FIG. 17A and illustrating the bus powerarrangement supplying power at an individual modular data pod;

FIG. 18 illustrates a server power supply according to one embodiment ofthe present disclosure that is a combination of the AC-DC converter ofFIG. 11 and the multi-output DC-DC converter of FIG. 13;

FIG. 19 illustrates a modular data pod according to one embodiment ofthe present disclosure, where each utility power feeder line hasseparate and distinct on-line single conversion DC UPSs connected tosingle conversion DC-DC converters (server supplies) at the input to theIT server loads;

FIG. 20A is a schematic illustration of one embodiment of the presentdisclosure (FIG. 19) of the bus power arrangement of a modular data podpowered by two utility power feeders in which on-line DC UPSs arelocated in between intermediate level buses and cable distributionbuses;

FIG. 20B is a continuation of the schematic illustration of FIG. 20A inwhich on-line single DC-DC power converters supply power directly to ITserver loads of a modular data pod;

FIG. 21 illustrates the main components of a modular data modular datapod having a UPS and single conversion DC-DC converters supplying powerto IT server loads according to one embodiment of the presentdisclosure;

FIG. 22 illustrates a comparison between the bus power arrangement of amodular data center and a modular data pod farm in which a 13.8 KV or a6.6 KV bus feed is supplied to both a modular data center and themodular data pod farm according to one embodiment of the presentdisclosure; and

FIG. 23 is a more detailed schematic illustration of the bus powerarrangement of FIG. 22 which shows only the 13.8 KV or a 6.6 KV bus feedsupplied to the modular data pod farm in which a step-down transformeris included within the modular data pod farm.

DETAILED DESCRIPTION

When introducing elements of the present disclosure, the articles “a,”“an,” “the” and “said” are intended to mean that there are one or moreof the elements. The terms “comprising,” “including,” and “having” areintended to be inclusive and mean that there may be additional elementsother than the listed elements.

The present disclosure relates to a DC UPS and power supply system withDC input voltage for a server load that improves overall data centerefficiency as compared to data centers with AC UPSs and server powersupplies with AC inputs. The present disclosure also relates to amodular UPS system design that has lower invested capital (CAPEX) andoperating energy costs (OPEX). The capital cost of the modularkW-capacity DC UPS is significantly reduced in comparison to centralizedMW-capacity AC UPSs.

Load capacity utilization of the data center is high because of themodular design based on IT load requirements. Higher overall data centerefficiency is achieved because the system can be operated at full ITloads. The DC UPS system is compact in comparison to the conventional ACUPS systems because there is no inverter (DC-AC) section that leads tolower UPS losses. The efficiency in the DC-DC server power supply ishigher as there is no AC-DC converter, which leads to lower losses incomparison to typical server power supplies with AC inputs.

The server power supply with a DC input voltage according to the presentdisclosure uses a high-frequency zero-voltage switching (ZVS) techniquewith a compact high-efficiency planar transformer to improve powersupply efficiency and to make the server power supply compact. Also,low-voltage (LV) lithium-ion batteries can be used for the DC UPS. TheLV lithium-ion batteries may supply 12 V, 24 V, or 48 V, for example.Thus, there is no need for high-voltage (HV) lead-acid batteries as inAC UPSs. The topology according to the present disclosure can be usedfor Tier 2, Tier 3, and Tier 4 systems in an N+1 configuration.

The embodiments of the modular data pod system and the associatedelectrical systems of the present disclosure provide significantimprovements over conventional modular pod data centers and theirelectrical systems. Two different electrical topologies are describedherein.

The more efficient topology uses an alternating current (AC)uninterruptible Power Supply (UPS) in an energy saver mode such that ahybrid configuration results in on-line dual conversion DC powersupplies supplying power to IT server loads while off-line AC UPSsprovide power to CDU loads, as described below with reference to FIGS.9-13.

More particularly, FIG. 9 shows a data center 600 having modular DC UPSs800 a . . . 800 n. Data center 600 uses modular, off-line AC UPSs 75 a .. . 75 n in energy saver (ES) mode to supply power to mechanical CDUloads 65 a-1 . . . 65 n-1, which are supplied power from bus 180 a, andCDU loads 65 a-2 . . . 65 n-2, which are supplied power from bus 180 b(see FIGS. 10A and 10B). DC UPSs 800 a . . . 800 n supply power torespective IT loads 50 a . . . 50 n via respective DC-DC converters 900a . . . 900 n that electrically couple the respective DC UPSs 800 a . .. 800 n and the respective IT loads (servers) 50 a . . . 50 n. The ITloads 50 a . . . 50 n are disposed between alternating hot aisles 610 a. . . 610 n and cold aisles 620 a . . . 620 n-1.

FIGS. 10A and 10B show schematic diagrams of data center 600 accordingto one embodiment of the present disclosure. As compared to conventionaldata center 100 described above with respect to FIGS. 2 and 3, in whichbuses 190 a, 190 b connect each server 50 a . . . 50 n via AC-DCconverters 520 a . . . 520 n and DC-DC converters 510 a . . . 510 n,buses 190 a, 190 b now connect to each server 50 a . . . 50 n viarespective AC-DC converters 820 a . . . 820 n from AC inputs 830 a, 830b, 830 c and respective DC-DC converters 900 a . . . 900 n connecting toeach server 50 a . . . 50 n via outputs 840 a . . . 840 b from AC-DCconverters 820 a . . . 820 n.

FIG. 11 shows DC UPS 800 which includes energy storage device 810. Theenergy storage device 810 may be a low-voltage, lithium-ion battery oran ultracapacitor or a combination of a low-voltage, lithium-ion batteryand an ultracapacitor. For example, the energy storage device 810 mayprovide 12 V, 24 V, or 48 V. The DC UPS 800 includes an AC-DC converter820, which includes AC inputs 830 a . . . 830 c from each of the buses190 a, 190 b. The outputs 840 a, 840 b from AC-DC converter 820 aresupplied to energy storage device 810 at positive junction 850 a onoutput 840 a and at negative junction 850 b on output 840 b and to DC-DCconverter 900 of the server power supply.

FIG. 12 shows DC-DC converter 900, which receives as inputs the DCoutputs 840 a, 840 b from AC-DC converter 820 and which providesmultiple outputs 920 a . . . 920 d for supplying a plurality ofdifferent DC voltages to servers 50 a . . . 50 n (see FIG. 10B).

FIG. 13 is a schematic diagram of an exemplary DC-DC converter 900according to one embodiment of the present disclosure. A DC voltage issupplied from either energy storage device 810 or AC-DC converter 820(see FIG. 11) to the DC-DC converter 900 via DC outputs 840 a, 840 b(see FIG. 12). Capacitor Cin is connected in parallel with the energystorage device 810 and with four MOSFETs M1 . . . M4 electricallycoupled together in an H-bridge configuration, all in parallel with theDC outputs 840 a and 840 b. Safety switch S1 is electrically coupled inseries between the H-bridge energy storage device Min, 810 on thepositive terminal side of energy storage device 810 and DC output 840 ato isolate the server side represented by the MOSFETs M1 to M4 and thesupply side of the bridge energy storage device Min, 810. MOSFETs M1 . .. M4 enable high-frequency switching of the server power supply 900 inthe range of about 80 kHz to about 120 kHz.

The MOSFETs M1 . . . M4 are controlled by digital signal processing(DSP) controller 1030 via gate drivers 1010. Based on input to DSPcontroller 1030, ZVS controller 1020 implements a zero-voltage switching(ZVS) strategy on MOSFETs M1 . . . M4 via gate drivers 1010. A DSP-basedZVS strategy may be employed to operate the server power supply 900. Thehigh-frequency DC-DC converter 900 thus includes the plurality ofMOSFETs M1 . . . M4 and zero-voltage switching controller 1020electrically coupled to the plurality of MOSFETs M1 . . . M4 to output aplurality of DC voltages V20 a, V20 b, V20 c, and V20 d at respective DCoutputs 920 a, 920 b, 920 c, and 920 d from the high-frequency DC-DCconverter 900.

An example of a ZVS control strategy is disclosed in “Evaluation of aNovel Analog Bi-directional ZVS Controller for High Frequency IsolatedDC-DC Converters,” by Subrata K. Mondal, published in the 34th AnnualConference of the IEEE Industrial Electronics Society, Nov. 10-13, 2008,E-ISBN 978-1-4244-1766-7, ©2008 IEEE, which is hereby incorporated byreference.

When operated, the four MOSFETs M1 . . . M4 output a first voltage V1 atmultiple input coils 9321, 9322, 9323, 9324 of single primary coil 932of multiple secondary winding planar transformer 930. First input coil9321 induces a voltage V2 a in first secondary coil 9341. Thus, a firstsub-transformer 9361 is formed by first input coil 9321 and firstsecondary coil 9341. Similarly, second input coil 9322 induces a voltageV2 b in second secondary coil 9342 such that a second sub-transformer9362 is formed by second input coil 9322 and second secondary coil 9342.

Additionally, third input coil 9323 induces a voltage V2 c in thirdsecondary coil 9343 such that a third sub-transformer 9363 is formed bythird input coil 9323 and third secondary coil 9343. Fourth input coil9324 induces a voltage V2 d in fourth secondary coil 9344 such that afourth sub-transformer 9364 is formed by fourth input coil 9324 andfourth secondary coil 9344.

Voltage V2 a is then supplied through diodes D1 . . . D4 and two LCfilters, which include inductors L1, L2 and capacitors C1, C2, whichresults in output DC voltage V20 a, 920 a. Voltage V2 b is then suppliedthrough diodes D5 . . . D8 and two LC filters, which include inductorsL3, L4 and capacitors C3, C4, which results in output DC voltage V20 b,920 b. Voltage V2 c is then supplied through diodes D9 . . . D12 and twoLC filters, which include inductors L5, L6 and capacitors C5, C6, whichresult in output DC voltage V20 c, 920 c. Voltage V2 d is then suppliedthrough diodes D13 . . . D16 and two LC filters, which include inductorsL7, L8 and capacitors C7, C8, which results in output DC voltage V20 d,920 d.

If output from AC-DC converter 820 is not available due to theunavailability of an AC input, then voltage Vin, from energy storagedevice 810 is switched on via safety switch S1 to the planar transformer930. Normally, the DC output of AC-DC converter 820 is slightly higheras it charges energy storage device 810 in normal operation and providesoutput at DC outputs 840 a and 840 b. Normally, since the output 840 aand 840 b of the AC-DC converter 820 and the output of the energystorage device 810 are in parallel so the voltage is the same at thecommon junction or terminal 850 b (see FIG. 11).

Some advantages of the data center 600 illustrated in FIGS. 10A and 10Bover conventional data centers, such as data center 100 illustrated inFIGS. 2 and 3, is that data center 600 eliminates higher capacity,over-sized, centralized, double conversion AC-DC/DC-AC AC UPSs such asAC UPS 40 in FIG. 1A. Furthermore, there is no need for lead-acidhigh-voltage batteries and a separate battery room. Electrical lossesare also reduced and efficiency is improved because of the use of singleconversion AC-DC DC UPS 800. Also, the UPS cost is less and the loadutilization factor is higher due to the use of a modular UPS design, andelectrical losses are reduced and efficiency improved because of the useof single-stage DC-DC conversion of server power supply with DC inputvoltage. Furthermore, efficiency of the server power supply is improveddue to use of the ZVS technique with super-compact, high switchingfrequency, and high efficiency planar transformer 930 of FIG. 13. Thedata center 600 also uses single stage conversion unlike a server powersupply with an AC input. Also, a low-voltage energy storage device suchas a battery or an ultra-capacitor or a combination of a battery and anultra-capacitor is used for the DC UPS, and Lithium-ion batteriesimprove battery life and energy density is higher in comparison tolead-acid batteries.

The design of data center 600 eliminates issues associated with a doubleconversion AC UPS 40 and the double conversion server power supply withthe AC input voltage via single phase AC supply lines 540 a, 540 b ofFIG. 5.

The use of modular DC UPSs 800 and DC-DC server power supplies 900 withDC input voltage, as illustrated in FIG. 10B, leads to higherefficiency, lower capital investment and lower operating energy costs.This topology can also be easily scaled based on the required load. TheDC UPSs 800 are of modular design based on initial data center load toimprove UPS utilization load factor which leads to higher efficiency andto reduced capital cost. The developed UPS system will occupy less floorspace due to its modular design. The Data Center design can be used forTier-2, Tier-3, and Tier-4 in an N+1 configuration.

Similar to the data center 600 of FIG. 9, FIG. 14 illustrates a modulardata center system 74 including a modular, scalable, double-conversionUPS system 76 and server power supplies with AC input voltages similarto the data center system 70 of FIG. 1B and which includes off-line ACUPSs 75 a . . . 75 n and 85 a . . . 85 n configured to operate in energysaver (ES) mode, which enables the modular scalable UPS system to have areduced PUE, instead of double-conversion mode as shown in FIG. 4 andserver power supplies 45 a . . . 45 n with AC input voltages. In ESmode, utility power 20 is supplied directly to IT loads 50 a . . . 50 nand mechanical CDU loads 65 a . . . 65 n. Line filtering is performed byfilter circuits (not shown) included with AC UPSs 75 a . . . 75 n. Theeffective efficiency of the AC UPSs 75 a . . . 75 n to the IT loads 50 a. . . 50 n and AC UPSs 85 a . . . 85 n to respective CDU loads 65 a . .. 65 n is about 99% in ES mode.

However, data center 74 of FIG. 14 differs from data center 600 of FIG.9 in that DC UPSs 800 a . . . 800 n are AC UPSs 75 a . . . 75 n andDC-DC converters 900 a . . . 900 n are AC-DC/DC-DC server power supplies45 a . . . 45 n.

In contrast, the efficiency of UPSs 145 a, 145 b (see FIG. 2) indouble-conversion mode at nominal load is around 94%-96%. Therefore,overall efficiency improvement in using the AC UPSs 75 a . . . 75 n inESS mode is about 3 to 5%. However, AC-DC and DC-DC converters in theserver power supplies 45 a . . . 45 n are required to supply a DCvoltage to respective server loads 50 a . . . 50 n. The lead-acidbatteries in the AC UPSs 75 a . . . 75 n have to be stored in a roomseparate from the servers 50 a . . . 50 n.

Referring to FIG. 15, efficiency improvements of the modular data pod1500 are achieved by connecting an AC UPS 1505 in energy saver (ES)mode, i.e., in off-line UPS mode. In ES mode, utility power is supplieddirectly to critical IT loads 1560 a . . . 1560 n and mechanical load1570 (e.g., a cooling mechanism) during normal operation. Line filteringis performed by using filter circuits (not shown) of the AC UPS 1505.For the modular data pod 1500 illustrated schematically in the blockdiagram of FIG. 15, the effective UPS efficiency is around 99%. UPSefficiency in double-conversion mode is around 94%-96% at nominal load.Therefore, overall efficiency improvement in ES mode is around 3 to 5%.In FIG. 15, the same UPS 1505 supplies power to both mechanical loads1570 and IT loads 1560 a . . . 1560 n.

The AC UPS 1505 includes an AC-DC converter 1520, a DC-DC converter1530, a DC-AC inverter 1540, an energy storage device 1535, and a StaticTransfer Switch (STS) 1545. The STS 1545 is in parallel with the AC-DCconverter 1520 and the DC-AC inverter 1540 of AC UPS 1505.

The IT cooling capacity of the CDU 1570 is around 350 kW. Therefore, theUPS rating should be around that range for optimum operation. Eitherutility power feeder line 1510 or 1515 (Utility A and B, respectively)has sufficient capacity to handle the entire load alone in case offailure of the other. However, each feeder line A and B 1510, 1515generally works in combination, each carrying 50% of the entire load.Feeder line B (1510) has a UPS in ES mode with 3 to 5 minutes of back-upstorage capacity. Additionally, the feeder line B (1510) is connecteddirectly to the CDU load 1570. Moreover, server power is supplied to theIT loads 1560 a . . . 1560 n via dual conversion AC-DC/DC-DC serverpower supplies 1550 a . . . 1550 n, where the server power may be in therange of 350 kW of IT load capacity. The UPS may be performed on onefeeder (e.g., feeder 1510) or both feeders (1510, 1515) depending onmodular data pod design requirements.

FIG. 16 illustrates modular data center 1600, where each utility powerfeeder line A and B (1510, 1515) has separate and distinct UPSs 1505,1605 in ES mode. The UPS 1505 includes an AC-DC converter 1520, a DC-DCconverter 1530, a DC-AC inverter 1540, an energy storage device 1535,and a Static Transfer Switch (STS) 1545. The UPS 1605 includes an AC-DCconverter 1620, a DC-DC converter 1630, a DC-AC inverter 1640, an energystorage device 1635, and a Static Transfer Switch (STS) 1645. Bothfeeder lines A and B (1510, 1515) are connected directly to cooling load1670. Moreover, server power is supplied to the IT loads 1560 a . . .1560 n via the server power supplies 1550 a . . . 1550 n, where theserver power may be in the range of about 350 kW of IT load capacity.

FIGS. 17A and 17B illustrate a one-line diagram of data center pods1700A, 1700B, having AC UPSs (A and B; 1764, 1768; see FIG. 17A) on bothbranch lines A, B and server power supply with AC input voltage. Duringnormal operation, the IT loads and the mechanical loads of data center1700 are powered entirely either by the utility power feeders 1 or 2(1710, 1720) as AC UPSs A, B (1764, 1768) are in Energy Saver (ES) mode.ES mode enables reduced PUE as the effective efficiency of an AC UPS inES mode is about 99%. The utility power source is usually connected tothe data center 1700 through the first utility power feeder 1 (1710).The second utility power feeder 2 (1720) is normally open (NO) viaswitch 1722 and supplies the data center load (both IT and mechanicalloads) in case the first utility power feeder 1 (1710) malfunctions.

A transformer 1740 steps down the voltage from the utility power feeders1 and 2 (1710, 1720) through bus tie 1732 to main switchgear 1750. Autility switchgear 1730 is positioned between the feeder lines 1710,1720 and the transformer 1740. Additionally, a main switchgear 1750 anda critical input switchgear 1760 are electrically coupled through tie1752 and are positioned between the transformer 1740 and the AC UPSs A,B (1764, 1768). AC UPS A (1764) includes an energy storage device 1766electrically coupled to tie 1761 between switchgear 1760 and splitfeeder 17611 to bus 1770 and split feeder 17612 to bus 1780, whereas ACUPS B (1768) includes an energy storage device 1769 electrically coupledto tie 1762 between switchgear 1760 and split feeder 17621 to bus 1770and split feeder 17622 to bus 1780.

Bus 1770 is electrically coupled to critical distribution switchgear1772 via tie 17711 and to switchgear 1782 via tie 17811. Bus 1780 iselectrically coupled to critical distribution switchgear 1772 via tie17712 and to switchgear 1782 via tie 17812.

Critical distribution switchgear 1772 is electrically coupled tomechanical load transformer 1786 via tie 1773 and split tie 17731 and tocableway “C-A” AC Distribution 1795 via bus 1794 via tie 1773 and splittie 17731.

Similarly, critical distribution switchgear 1782 is electrically coupledto mechanical load transformer 1788 via tie 1783 and split tie 17831 andto cableway “C-B” AC Distribution 1797 via bus 1796 and tie 1783 andsplit tie 17832.

In operation or use, when a disturbance in the utility power occurs, theUPSs A, B (1764, 1768) immediately take over the loads. When thedisturbance in the utility power feeder 1 (1710) is no longer present,the loads are shifted from the UPS batteries (1766, 1769) to the utilitypower feeder 1 (1720). The UPS batteries 1766, 1769 are recharged by theutility power feeder via step-down transformer 1740. An auxiliarydistribution source 1758 supplies power to the loads upon failure of theUPSs A and B (1764, 1768). The power is supplied from auxiliarydistribution source 1758 via ties 1759 to and split tie 17591 to outputswitchgear 1770 and via split tie 17592 to output switchgear 1780.

Mechanical load transformers (1786, 1788) supply power to the mechanical(CDU) load via switch gears 1790, 1792. Either transformer 1786, 1788has sufficient capacity to handle the entire mechanical load alone incase of failure of the other. If either transformer 1786, 1788 fails,then power is supplied to cableways M-A or M-B (1791, 1793) viacross-tie 1798 between switchgear 1790 and 1792. Mechanical loadtransformers 1786, 1788 are connected to the UPSs A and B (1764, 1768)via first critical distribution switchgear 1772 and second criticaldistribution switchgear 1782, as well as output switchgears 1770 and1780, as shown in FIG. 17A.

Either cableway 1795, 1797 (C-A or C-B) has sufficient capacity tohandle the entire IT load alone in case of failure of the other.However, cableways 1791, 1793, 1795, 1797 generally work in combination,each carrying 50 % of the load. However, cableways C-A and C-B (1795,1797) usually supply power to IT loads, whereas mechanical cableways M-Aand M-B (1791, 1793) usually supply power to mechanical/CDU loads 1749.Bus 1794 to cableway C-A (1795) can be cross-tied to bus 1796 andcableway C-B (1797) via cross-tie 1799.

The cooling mechanism is in thermal communication with each of theplurality of servers or IT loads 1723, 1725, 1729, 1731, 1733, andcirculates a coolant that removes heat generated by the plurality ofservers or IT loads 1723, 1725, 1729, 1731, 1733. The coolant may bepumped by a very efficient cooling distribution unit (CDU) with a heatexchanger (not shown) that allows the data center 1700C to userefrigerant cooling. Each CDU may support approximately 350 kW of ITload capacity.

FIG. 17B illustrates connections between buses M-A, M-B (1747, 1751) andmechanical/CDU loads 1749, and between buses C-A, C-B (1709, 1745) andIT loads. IT loads include a plurality of server rack assemblies (1723,1725, 1729, 1731, 1733), which are separated from each other via coldaisles 1727. Additionally, the plurality of server rack assemblies(1723, 1725, 1729, 1731, 1733) are enclosed within hot aisles 1721,1735. Each server rack assembly (1723, 1725, 1729, 1731, 1733) iselectrically coupled to cableways C-A, C-B (1709, 1745) via server powersupplies 1800 (see FIGS. 17B and 18).

Referring to FIG. 18, each server power supply 1800 includes an AC-DCconverter 1810 and multi-output DC-DC converter 1820, as describedbelow. Server power supply 1800 as illustrated in FIG. 17B, includes twosingle phase AC inputs 1805 a, 1805 b from respective feed buses 1709 or1745. The AC-DC converter 1810 converts a single-phase (e.g., 277 V) ACvoltage to an intermediate DC voltage. DC-DC converter 1820 converts theintermediate DC voltage into multiple low DC voltages 1825 a . . . 1825d. For example, DC-DC converter 1820 can supply approximately 3.3 VDC, 5VDC, 12 VDC and −12 VDC to the server as shown.

FIG. 19 illustrates a second topology using Hybrid (both AC and DC)distribution wherein a direct current (DC) uninterruptible Power Supply(UPS) in conjunction with a server power supply with a DC input isemployed, as described below with reference to FIGS. 19-24.

As compared to the server power supply 1800 of FIG. 18, modular data pod1900 illustrated in FIG. 19 includes an on-line single conversion AC-DCconverter 1930 and an energy storage device 1940 connected in ES mode1905 for utility feed line 1910 for Utility A and an on-line singleconversion AC-DC converter 1950 and an energy storage device 1960connected in ES mode 1907 utility feed line 1920 for Utility B.

The on-line single conversion AC-DC converter 1930 and energy storagedevice 1940 connected in ES mode 1905 correspond to single conversion DCUPS 1810 and energy storage device 1815 of FIG. 18. Similarly, on-linesingle conversion AC-DC converter 1950 and energy storage device 1960connected in ES mode 1907 also correspond to single conversion DC UPS1810 and energy storage device 1815 of FIG. 18.

On-line single conversion DC-DC converters 1980 a . . . 1980 n that aresupplied by on-line single conversion AC-DC converter 1930 and energystorage device 1940 and on-line single conversion AC-DC converter 1950and energy storage device 1960 and which supply power to IT loads 1990 a. . . 1990 n correspond to multi-output DC-DC converter 1820 and DCoutputs 1825 a, 1825 b, 1825 c and 1825 d in FIG. 18.

More particularly this further efficiency improvement is achieved byconnecting on-line DC UPS 1905, 1907 on both supply lines 1910 and 1920along with a server power supply with a DC input 1980 a . . . 1980 n, asillustrated in FIG. 19. The first DC UPS 1905 includes AC-DC converter1930 and energy storage device 1940. The second DC UPS 1907 includesAC-DC converter 1950 and energy storage device 1960. The modular datacenter 1900 is powered by a first utility feed 1910 and a second utilityfeed 1920. The server power is supplied from server power supplies 1980a . . . 1980 n via the DC UPSs 1905, 1907. The DC output of the serverpower is supplied to the IT loads 1990 a . . . 1990 n. The coolingmechanism 1970 (or mechanical load) is supplied by both utility feeds1910, 1920.

The DC UPS system 1900 is compact in comparison to existing AC UPSsystems because there is no inverter (DC-AC) section, which leads tolower UPS losses. The efficiency of the DC server power supplies 1980 a. . . 1980 n is also higher as there is no front-end AC-DC convertersection, which leads to lower losses in comparison to conventionalserver power supplies with AC inputs.

The server power supplies 1980 a . . . 1980 n, with a DC input voltage,according to one embodiment of the present disclosure, use thehigh-frequency zero-voltage switching (ZVS) technique with compacthigh-efficiency planar transformer 930, as described above with respectto FIG. 13, to improve power supply efficiency and to make the serverpower supplies 1980 a . . . 1980 n compact. Also, low-voltage (LV)Lithium-ion batteries can be used for the DC UPSs 1905, 1907. The LVlithium-ion batteries may supply 24 V or 48 V. Thus, there is no needfor high-voltage (HV) lead acid batteries as in AC UPSs. The topologyaccording to the present disclosure can be used for Tier 2, Tier 3, andTier 4 systems in an N+1 configuration.

FIGS. 20A and 20B are schematic diagrams of the bus power arrangementand data centers having modular DC UPS-A 820 a and DC UPS-B 820 b,respectively (see FIG. 11). DC UPSs A and B (820 a and 820 b) supplypower to respective IT loads via respective DC-DC converters 900, asshown in FIGS. 20B and 21. Buses C-A, C-B (2009, 2045) connect to eachserver via respective DC-DC converter 900 of server power supply 800.

In particular, DC UPS A, 820 a and DC-UPS B, 820 b in FIG. 20A are onboth feeder lines 2010, 2020 and supply power to the servers with a DCinput voltage.

During normal operation, the IT loads and the mechanical loads of datacenter 2000 are powered entirely either by the utility power feeders 1or 2 (2010, 2020) via on-line single conversion DC UPSs A, B (820 a, 820b). The utility power source is usually connected to the data center2000B through the first utility power feeder 1 (2010). The secondutility power feeder 2 (2020) is normally open (NO) via switch 2022 andsupplies the data center load (both IT and mechanical loads) in case thefirst utility power feeder 1 (2010) malfunctions. A transformer 2040steps down the voltage from the utility power feeders 1 and 2 (2010,2020) through bus tie 2032 to main switchgear 2050. A utility switchgear2030 is positioned between the feeder lines 2010, 2020 and thetransformer 2040. Additionally, a main switchgear 2050 and a criticalinput switchgear 2060 are electrically coupled through tie 2052 and arepositioned between the transformer 2040 and the DC UPSs A, B (820 a, 820b). DC UPS A 820 a includes an energy storage device 810 a, whereas DCUPS B 820 b includes energy storage device 810 b. Bus 2070 iselectrically coupled to critical distribution switchgear 2072 via tie20711 and to switchgear 2082 via tie 20811. Bus 2080 is electricallycoupled to critical distribution switchgear 2072 via tie 20712 and toswitchgear 2082 via tie 20812.

Critical distribution switchgear 2072 is electrically coupled tomechanical load transformer 2086 via tie 2073 and split tie 20731 and tocableway “C-A” DC Distribution 2095 via bus 2094 via tie 2073 and splittie 20731.

Similarly, critical distribution switchgear 2082 is electrically coupledto mechanical load transformer 2088 via tie 2083 and split tie 20831 andto cableway “C-B” DC Distribution 2097 via bus 2096 and tie 2083 andsplit tie 20832.

The IT loads (2023, 2025, 2029, 2031, 2033) are supplied via on-line DCUPS (820 a, 820 b). Each UPS, 820 a, 820 b has sufficient capacity tohandle the entire IT load alone in case of failure of the other.However, UPS 820 a, 820 b generally work in combination, each carrying50% of the load. The DC UPS batteries 810 a, 810 b are recharged by theutility power feeder via step-down transformer 2040. An auxiliarydistribution source 2058 supplies power to the loads upon failure of theDC UPSs A and B (820 a, 820 b). The power is supplied from auxiliarydistribution source 2058 via ties 2059 to and split tie 20591 to outputswitchgear 2070 and via split tie 20592 to output switchgear 2080.

Mechanical load transformers (2086, 2088) supply power to the mechanical(CDU) load 2049 (see FIG. 20B) via switch gears 2090, 2092. Eithertransformer 2086, 2088 has sufficient capacity to handle the entiremechanical load alone in case of failure of the other. If eithertransformer 2086, 2088 fails, then power is supplied to cableways M-A orM-B (2091, 2093) via cross-tie 2098 between switchgear 2090 and 2092.Mechanical load transformers 2086, 2088 are connected to the source viaa first critical distribution switchgear 2072 and a second criticaldistribution switchgear 2082, as well as output switchgears 2070 and2080.

Either cableway 2095, 2097 (C-A or C-B) has sufficient capacity tohandle the entire IT load alone in case of failure of the other.However, cableways 2095, 2097, 2091, and 2093 generally work incombination, each carrying 50% of the load. However, cableways C-A andC-B (2095, 2097) are usually used for IT loads, whereas mechanicalcableways M-A and M-B (2091, 2093) are usually used for mechanical/CDUloads 2049. Bus 2094 to cableway C-A (2095) can be cross-tied to bus2096 and cableway C-B (2097) via cross-tie 2099.

FIG. 20B illustrates connections between buses M-A, M-B (2047, 2051) andmechanical/CDU loads 2049, and between buses C-A, C-B (2009, 2045) andIT loads. IT loads include a plurality of server rack assemblies (2023,2025, 2029, 2031, 2033), which are separated from each other via coldaisles 2027. Additionally, the plurality of server rack assemblies(2023, 2025, 2029, 2031, 2033) are enclosed within hot aisles 2021,2035. Each server rack assembly (2023, 2025, 2029, 2031, 2033) iselectrically coupled to cableways C-A, C-B (2009, 2045) via server powersupplies 900 (see FIG. 20B).

FIG. 21 illustrates a modular data pod 2400 having a DC UPS 2410 thatincludes an AC-DC converter (not shown, but is part of the DC UPS 2410)and an energy storage device 2470. The energy storage device 2470 may bea low voltage, lithium-ion battery. For example, the battery may provide24 or 48 V. The DC UPS 2410 includes a plurality of AC inputs from eachof the buses 2072 and 2082 (see FIG. 20A). The outputs from the AC-DCconverter of DC UPS 2410 are supplied to the energy storage device 2470.

The server power supply 900 has been described above with respect toFIG. 12 and has been shown in FIGS. 20B and 21, which includes two DCinputs 840 a and 840 b from respective feed buses. The DC-DC converter900 supplies a plurality of different DC voltages to the ITloads/servers 2440. DC-DC converter 900 converts the input DC voltageinto multiple low DC voltages 920 a . . . 920 d. For example, DC-DCconverter 900 can supply approximately 3.3 VDC, 5 VDC, 12 VDC and −12VDC to the server as shown.

FIG. 21 illustrates the main components of modular data pod 2400according to one embodiment of the present disclosure. IT loads includea plurality of server rack assemblies 2440, which are separated fromeach other via a cold aisle 2444. Additionally, the plurality of serverrack assemblies 2440 are enclosed within hot aisles 2446. Each serverrack assembly 2440 is electrically coupled to buses C-A, C-B (2405,2255) via server power supplies 900 as described above.

Some advantages of the modular data center 2400 over prior art modulardata centers 1100, 1200 (FIGS. 6 and 7), is that data center 2400eliminates the need for less efficient double conversion AC UPSs.Furthermore, a high energy density, long life, Li-ion battery may beused in place of a low density, reduced life, lead-acid battery, asspace is an issue with modular data centers. Thus, efficiencyimprovement is achieved because of the use of an AC UPS in ES mode withAC distribution (Topology-1) or the use of a single conversion DC UPSwith Hybrid (both AC and DC) distribution (Topology-2). Furthermore,electrical losses are reduced and efficiency is improved because of theuse of the single-stage conversion of a server power supply with a DCinput voltage. Moreover, efficiency of the server power supply isimproved due to the use of a ZVS technique with compact high switchingand a high efficiency planar transformer. Also, a low-voltage batterymay be used for a DC UPS, such as a Lithium-ion battery, which improvesbattery life and energy density compared to lead-acid batteries.

The use of modular DC UPSs and server power supplies with DC inputvoltage leads to higher efficiency, lower capital investment and loweroperating energy costs. This topology can also be easily scalable basedon the required load. The UPS system disclosed herein occupies lessfloor space due to its modular design. The modular data center designscan be used for Tier-2, Tier-3, and Tier-4 in N+1 configurations.

The following Table shows an approximate percentage of electricalefficiency improvement at rated loads for the modular data center 2400.

at Rated Load AC UPS in on-line double AC UPS in Hybrid conversionEnergy Saver Distribution: mode & Server mode & Server DC UPS & Serverpower supply power supply power supply with AC input with AC input withDC input (Existing Art) (Topology-1) (Topology-2) UPS Crictical 95% 99%97.50% UPS Mechanical 95% 99% 99.00% PS - AC/DC 94% 94% Section PS -DC/DC 93% 93%   94% Section Overall 78.90%   85.68%   90.73% EfficiencyEfficiency 6.78%  11.84% Improvement

The power system according to the present disclosure may be used inlarge MW power applications by connecting them in parallel. The powersystem may also be used in applications across the full power spectrumfrom small to very large power applications. The urban data center isbenefited from this power system as it occupies less floor space.

FIG. 22 illustrates a comparison between the bus power arrangement ofthe modular data center 600 and a modular data pod farm 2500 in which13.8 KV or 6.6 KV bus feeds as utility feed lines 105 a and 105 b aresupplied to both the modular data center 600 and the modular data podfarm 2500 that includes modular data pods 1700B or 2000B according toone embodiment of the present disclosure to provide similar efficiencyimprovements to the prior art modular data center 600 as are applied tothe modular data farm 2500 via the embodiments of the presentdisclosure.

FIG. 23 is a more detailed schematic illustration of the bus powerarrangement of FIG. 22 which shows only the 13.8 KV or a 6.6 KV busfeeds 105 a or 105 b supplied only to the modular data pod farm 2500 inwhich step-down transformer 115 is included within the modular data podfarm 2500. The modular data pod farm 2500 includes cooling tower 2510and individual data pods 2400 a . . . 2400 n that are illustrated inFIG. 21 as modular data pod 2400.

The electrical systems and methods for the modular data centers andmodular data pods according to the present disclosure are furtherdescribed in “Efficient Data Center design using Novel Modular DC UPS,Server Power supply with DC voltage and Modular CDU cooling,” by SubrataMondal and Earl Keisling, published in the 2012 IEEE InternationalConference on Power Electronics, Drives and Energy Systems, Dec. 16-19,2012, Bengaluru, India, E-ISBN 978-1-4673-4508-8, ©2012 IEEE, which ishereby incorporated by reference.

While several embodiments of the disclosure have been shown in thedrawings and/or discussed herein, it is not intended that the disclosurebe limited thereto, as it is intended that the disclosure be as broad inscope as the art will allow and that the specification be read likewise.Therefore, the above description should not be construed as limiting,but merely as exemplifications of particular embodiments. Those skilledin the art will envision other modifications within the scope and spiritof the claims appended hereto.

What is claimed is:
 1. A modular system for supplying DC power to atleast one server, comprising: a DC uninterruptible power supply (UPS)including: an AC-DC converter; and an energy storage device electricallycoupled to an output of the AC-DC converter; and a DC-DC converterdirectly connected to the output of the AC-DC converter of the DC UPS.2. The modular system according to claim 1, wherein the energy storagedevice is a low-voltage battery.
 3. The modular system according toclaim 1, wherein the low-voltage battery is a 12 V battery, a 24 Vbattery, or a 48 V battery.
 4. The modular system according to claim 1,wherein the energy storage device is a lithium-ion battery or acombination of a lithium-ion battery and a ultra-capacitor.
 5. Themodular system according to claim 1, wherein the DC-DC convertersupplies a plurality of DC voltages to the at least one server.
 6. Themodular system according to claim 1, wherein a single power conversionis performed between the energy storage device and the server.